Face recognition system, face recognition method, and storage medium

ABSTRACT

A face recognition system, a face recognition method, and a storage medium that can perform face matching smoothly in a short time are provided. The face recognition system includes: a face detection unit that detects a face image from an image including an authentication subject as a detected face image; a storage unit stores identification information identifying the authentication subject and a registered face image of the authentication subject in association with each other; and a face matching unit that, in response to acquisition of the identification information identifying the authentication subject, matches, against the registered face image corresponding to the acquired identification information, the detected face image detected by the face detection unit from an image captured before the acquisition.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.17/199,814, filed on Mar. 12, 2021, which is a continuation of U.S.patent application Ser. No. 16/288,231, filed on Feb. 28, 2019, whichissued as U.S. Pat. No. 10,977,483, which is a continuation of U.S.patent application Ser. No. 16/079,814, filed on Aug. 24, 2018, whichissued as U.S. Pat. No. 11,055,513, which is a National Stage ofInternational Application No. PCT/JP2017/006861 filed on Feb. 23, 2017,which claims priority from Japanese Patent Application No. 2016-036406,filed Feb. 26, 2016, the contents of all of which are incorporatedherein by reference in their entirety.

TECHNICAL FIELD

The present invention relates to a face recognition system, a facerecognition method, and a storage medium.

Background Art

In recent years, biometric authentication that performs authenticationusing biometric information that is information on a physical feature orbehavior feature of a human has been utilized in a situation of identityverification. Face authentication that is one of the forms of biometricauthentication is advantageous because of less mental stress at anauthentication subject, ability of authentication from a distant place,a mental deterrent effect against a fraud, or the like.

Face authentication technologies have been utilized for identityverification in various fields. For example, in a gate system installedin an entrance gate of a facility such as a theme park, a faceauthentication technology is utilized for identity verification ofvisitors who use a ticket such as annual pass thereof or the like (NonPatent Literature 1).

CITATION LIST Non Patent Literature

NPL 1: NEC Corporation, “Face Authentication: Gate System”, [online],[searched on Feb. 12, 2016], Internet <URL:http://jpn.nec.com/ad/usj/entry.html>

SUMMARY OF INVENTION Technical Problem

In the face authentication technology in the conventional gate system,it is necessary for the staff of a facility to operate a camera tocapture a face image of a visitor for face matching after ticketpresentation in which the visitor causes a reading unit of a gateapparatus to read information on the ticket. In this way, since theconventional gate system requires the operation of capturing a faceimage of a visitor after the visitor causes information on a ticket tobe read and before the visitor enters the facility, many visitorswaiting for identity verification by face authentication may be detainedat the entrance gate of the facility.

The present invention intends to provide a face recognition system, aface recognition method, and a storage medium that can perform facematching smoothly in a short time.

Solution to Problem

According to one example aspect of the present invention, provided is aface recognition system including: a face detection unit that detects aface image from an image including an authentication subject as adetected face image; a storage unit that stores identificationinformation identifying the authentication subject and a registered faceimage of the authentication subject in association with each other; anda face matching unit that, in response to acquisition of theidentification information identifying the authentication subject,matches, against the registered face image corresponding to the acquiredidentification information, the detected face image detected by the facedetection unit from an image captured before the acquisition.

According to another example aspect of the present invention, providedis a face recognition method including: detecting a face image from animage including an authentication subject as a detected face image; andin response to acquisition of the identification information identifyingthe authentication subject, matching, against the registered face imageassociated with the acquired identification information, the detectedface image detected from an image captured before the acquisition.

According to yet another example aspect of the present invention,provides is a storage medium in which a program is stored, and theprogram causes a computer to execute: detecting a face image from animage including an authentication subject as a detected face image; andin response to acquisition of the identification information identifyingthe authentication subject, matching, against the registered face imageassociated with the acquired identification information, the detectedface image detected from an image captured before the acquisition.

Advantageous Effects of Invention

According to the present invention, face matching can be performedsmoothly in a short time.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram illustrating a face recognition systemaccording to a first example embodiment of the present invention.

FIG. 2 is a block diagram illustrating a functional configuration of theface recognition system according to the first example embodiment of thepresent invention.

FIG. 3 is a schematic diagram illustrating an example of an imagecaptured by a fixed camera in the face recognition system according tothe first example embodiment of the present invention.

FIG. 4 is a schematic diagram illustrating an example of various datafor detected face images temporarily stored in a storage unit of a facematching apparatus in the face recognition system according to the firstexample embodiment of the present invention.

FIG. 5 is a schematic diagram illustrating an example of a process ofN:1 matching in the face recognition system according to the firstexample embodiment of the present invention.

FIG. 6 is a block diagram illustrating an example of a hardwareconfiguration of a face matching apparatus in the face recognitionsystem according to the first example embodiment of the presentinvention.

FIG. 7 is a block diagram illustrating an example of a hardwareconfiguration of a datacenter server in the face recognition systemaccording to the first example embodiment of the present invention.

FIG. 8 is a sequence diagram illustrating a face recognition methodaccording to the first example embodiment of the present invention.

FIG. 9 is a flowchart (part 1) illustrating the face recognition methodaccording to the first example embodiment of the present invention.

FIG. 10 is a flowchart (part 2) illustrating the face recognition methodaccording to the first example embodiment of the present invention.

FIG. 11 is a schematic diagram illustrating another example of an imagecaptured by a fixed camera in the face recognition system according tothe first example embodiment of the present invention.

FIG. 12 is a block diagram illustrating a functional configuration of aface recognition system according to a second example embodiment of thepresent invention.

FIG. 13 is a schematic diagram illustrating an example of a process ofN:1 matching in the face recognition system according to the secondexample embodiment of the present invention.

FIG. 14A is a schematic diagram illustrating an example of a differentimage captured by a fixed camera in a face recognition system accordingto a third example embodiment of the present invention.

FIG. 14B is a schematic diagram illustrating an example of a differentimage captured by a fixed camera in the face recognition systemaccording to the third example embodiment of the present invention.

FIG. 15 is a block diagram illustrating a functional configuration of aface recognition system according to the third example embodiment of thepresent invention.

FIG. 16 is a flowchart illustrating automatic update of a registeredface image in the face recognition system according to a fourth exampleembodiment of the present invention.

FIG. 17A is a schematic diagram illustrating various data in automaticupdate of a registered face image in the face recognition systemaccording to the fourth example embodiment of the present invention.

FIG. 17B is a schematic diagram illustrating various data in automaticupdate of a registered face image in the face recognition systemaccording to the fourth example embodiment of the present invention.

FIG. 17C is a schematic diagram illustrating various data in automaticupdate of a registered face image in the face recognition systemaccording to the fourth example embodiment of the present invention.

FIG. 18 is a flowchart illustrating a face recognition method accordingto a fifth example embodiment of the present invention.

FIG. 19 is a flowchart illustrating a face recognition method accordingto a sixth example embodiment of the present invention.

FIG. 20 is a schematic diagram illustrating an example of a matchingwindow displayed on a display unit in the face recognition methodaccording to the sixth example embodiment of the present invention.

FIG. 21 is a block diagram illustrating a functional configuration of aface recognition system according to a seventh example embodiment of thepresent invention.

FIG. 22 is a schematic diagram illustrating a face recognition systemaccording to an eighth example embodiment of the present invention.

FIG. 23 is a block diagram illustrating a functional configuration ofthe face recognition system according to the eighth example embodimentof the present invention.

FIG. 24 is a flowchart illustrating a face recognition method accordingto the eighth example embodiment of the present invention.

FIG. 25 is a schematic diagram illustrating an example of a computerapparatus.

FIG. 26 is a black diagram illustrating a functional configuration of aface recognition system according to another example embodiment of thepresent invention.

FIG. 27 is a black diagram illustrating a functional configuration of aface matching apparatus according to another example embodiment of thepresent invention.

FIG. 28 is a black diagram illustrating a functional configuration of aface recognition system according to yet another example embodiment ofthe present invention.

DESCRIPTION OF EMBODIMENTS First Example Embodiment

A face recognition system and a face recognition method according to afirst example embodiment of the present invention will be described byusing FIG. 1 to FIG. 11 .

First, the face recognition system according to the present exampleembodiment will be described by using FIG. 1 to FIG. 7 . FIG. 1 is aschematic diagram illustrating a face recognition system according tothe present example embodiment. FIG. 2 is a block diagram illustrating afunctional configuration of the face recognition system according to thepresent example embodiment. FIG. 3 is a schematic diagram illustratingan example of an image captured by a fixed camera in the facerecognition system according to the present example embodiment. FIG. 4is a schematic diagram illustrating an example of various data fordetected face images temporarily stored in a storage unit of a facematching apparatus in the face recognition system according to thepresent example embodiment. FIG. 5 is a schematic diagram illustratingan example of a process of N:1 matching in the face recognition systemaccording to the present example embodiment. FIG. 6 is a block diagramillustrating an example of a hardware configuration of a face matchingapparatus in the face recognition system according to the presentexample embodiment. FIG. 7 is a block diagram illustrating an example ofa hardware configuration of a datacenter server in the face recognitionsystem according to the present example embodiment.

The face recognition system according to the present example embodimentperforms identity verification by using face matching at the entrancegate of a facility, where an authentication subject is a visitor whointends to enter the facility by using an admission ticket. For example,the facility may be a theme park, an event hall, a stadium, a concerthall, or the like. For example, a ticket used by a visitor is anadmission ticket called an annual pass, an annual passport, or the likewith which the visitor can enter the facility any times during aparticular period such as a year or the like, although the type thereofis not limited in particular. The admission ticket may be a paper ticketor an electronic ticket as long as it is a medium in whichidentification information that identifies the admission ticket isrecorded in a readable manner. A case of identity verification by usingface matching when a visitor uses an annual pass to enter a facilitywill be described below.

As illustrated in FIG. 1 and FIG. 2 , the face recognition system 1according to the present example embodiment includes a gate apparatus10, a fixed camera 20, a face matching apparatus 30, and a datacenterserver 40. The gate apparatus 10, the fixed camera 20, and the facematching apparatus 30 are installed at an entrance gate 50 of afacility. On the other hand, the datacenter server 40 is installedwithin a datacenter located in a remote place of the entrance gate 50,for example.

At the entrance gate 50 where the gate apparatus 10 and the like areinstalled, a roof 502 is installed. A lighting apparatus 504 is providedto the roof. Further, a guide plate 508 indicating the entrance gate 50is provided to the roof 502 so as to be located above the gate apparatus10.

The face matching apparatus 30 and the datacenter server 40 areconnected to a network via a network 60, respectively, and can becommunicated with each other via the network 60. The network 60 may be aWide Area Network (WAN) or a Local Area Network (LAN), for example,although the type thereof is not limited in particular.

Further, the gate apparatus 10 and the fixed camera 20 are directly,locally connected in a communicable manner to the face matchingapparatus 30 through cable connection or the like, respectively. Theconnection among the gate apparatus 10, the fixed camera 20, and theface matching apparatus 30 may be of a wired scheme or a wirelessscheme.

An annual pass can be purchased from a web ticket store or a ticketbooth. A web server 70 that provides a web ticket store and a ticketbooth terminal 80 are connected to the network 60. The web server 70 andthe ticket booth terminal 80 can communicate with the datacenter server40 via the network 60, respectively. The web server 70 is installedinside a datacenter located in a remote place of the entrance gate 50,for example. The ticket booth terminal 80 is installed inside the ticketbooth neighboring the entrance gate 50, for example.

Next, each component of the face recognition system 1 according to thepresent example embodiment will be described in detail.

The gate apparatus 10 has a main unit 102, a fence 104, a gate 106, areading unit 108, a hand camera 110, and a gate control unit 112.

The main unit 102 and the fence 104 are installed so as to face eachother. A path 114 through which a visitor walks to enter the facilityruns between the main unit 102 and the fence 104. On the path 114, anentrance 114 a is outside, and an exit 114 b is inside. The main unit102 is installed on the right side when viewed from the entrance 114 ato the exit 114 b of the path 114. On the other hand, the fence 104 isinstalled on the left side when viewed from the entrance 114 a to theexit 114 b of the path 114.

The gate 106 is provided on the sidewall on the main unit 102 on thepath 114 so as to block the path 114 during a standby state. When openedfrom a closed state during a standby state for blocking the path 114,the gate 106 allows a visitor to walk through the path 114 and enter theinside of the facility. The gate 106 is a turn-style gate in which threebars rotate, for example. Note that, without limited to the above,various types of gates may be used as a gate. For example, as the gate106, a flapper gate in which two flappers provided on both sides or asingle flapper provided on one side of the path 114 is opened and closedmay be used.

As described later, the gate 106 is opened when identity verification byface matching is successful. Thereby, the visitor is allowed to walkthrough the path 114 and enter the inside of the facility.

Note that the gate 106 may be a gate that is in an opened state during astandby state and maintains the opened state when identity verificationby face matching is successful, and is closed when identity verificationby face matching is failed.

The reading unit 108 is provided on a portion on the entrance 114 a sideon the path 114 of the gate 106 on the top of the main unit 102. Thereading unit 108 reads information recorded in an annual pass carried bya visitor from the annual pass. Specifically, in an annual pass,identification (ID) information that is identification informationuniquely identifying the annual pass is recorded. The reading unit 108reads ID information from an annual pass. The ID information read by thereading unit 108 may be a member number, a serial number, or the like ofthe annual pass, for example. An annual pass is a medium that is carriedby a visitor who is an authentication subject and required when thevisitor enters the inside of the facility and on which ID informationuniquely identifying itself is recorded. Here, the medium may be amedium, such as a card, a sheet, a smartphone, or the like, which hasinformation identifying an authentication subject. As described later,information on purchasers who have purchased the annual passes inassociation with ID information of the annual passes is accumulated inthe datacenter server 40.

The reading unit 108 has a reading scheme in accordance with a recordingscheme of ID information on an annual pass. For example, when an annualpass has ID information recorded in a one-dimensional code such as abarcode or a two-dimensional code such as a QR code (registeredtrademark), the reading unit 108 is a code reader such as a barcodereader, a QR code reader, or the like. Further, for example, when anannual pass has ID information recorded in a non-contact IC card or anon-contact IC tug using Radio Frequency Identification (RFID), thereading unit 108 is an RFID reader.

When there is ticket presentation of an annual pass at the reading unit108, the reading unit 108 reads ID information recorded in the annualpass from the annual pass. Note that ticket presentation here means thata visitor who is an authentication subject causes the reading unit 108to read information including the ID information recorded in an annualpass.

The reading unit 108 transmits ID information read from an annual passto the gate control unit 112 described later. Note that identificationinformation identifying an authentication subject is not limited to IDinformation stored in a medium such as an annual pass. Theidentification information identifying an authentication subject mayinclude biometric information of an authentication subject such as afinger print, a vein, an iris, or the like, for example, and may be anyinformation that can identify an authentication subject. In this case,the reading unit 108 may be a finger print scanner, a vain scanner, acamera, or the like that can read biometric information such as a fingerprint, a vein, an iris, or the like of an authentication subject.

The hand camera 110 is provided to a portion near the gate 106 on thetop of the main unit 102. The hand camera 110 is a digital video camera,for example, and can capture a face image of a visitor who is anauthentication subject and acquire the face image according to anoperation of the staff of the facility. The hand camera is otherimage-capturing units used when face matching based on a face imagecaptured by the fixed camera 20 is failed, as described later. Note thatthe hand camera 110 may be any camera as long as it can acquire a faceimage of a visitor and may be a digital still camera.

The hand camera 110 transmits image data of the captured face image of avisitor to the gate control unit 112 described later.

The gate control unit 112 controls the operation of the gate apparatus10. The reading unit 108 is connected to the gate control unit 112 so asto be able to communicate therewith. Further, the gate 106 is connectedto the gate control unit 112 in a controllable manner. Further, the handcamera 110 is connected to the gate control unit 112 so as to be able tocommunicate therewith.

ID information of an annual pass read by the reading unit 108 istransmitted to the gate control unit 112 from the reading unit 108. Thegate control unit 112 transmits ID information of an annual passtransmitted from the reading unit 108 to the face matching apparatus 30.

Further, the gate control unit 112 controls opening and closing of thegate 106 based on a matching result signal transmitted from the facematching apparatus 30 described later.

Further, image data of a face image of a visitor captured by the handcamera 110 is transmitted to the gate control unit 112 from the handcamera 110. The gate control unit 112 transmits image data of a faceimage transmitted from the hand camera 110 to the face matchingapparatus 30.

The fixed camera 20 is fixed to the upper end of a support pillar 202installed inside the facility with respect to the gate apparatus 10. Thefixed camera 20 is an image capturing unit that captures an image of anarea in front of the gate apparatus 10 and in which the orientationfacing the side of the outside of the facility is fixed. The fixedcamera 20 is fixed at a height located above a head of a human of aheight of around 200 cm, for example, from the ground face at theentrance gate 50 and is directed obliquely downward to face an area infront of the gate apparatus 10. Note that a fixing scheme of the fixedcamera 20 is not limited to a scheme using the support pillar 202. Forexample, the fixed camera 20 may be hanged from and fixed to the roof502 of the entrance gate 50.

The fixed camera 20 fixed as described above captures an image of anarea in front of the gate apparatus 10 that is the entrance side to theinstallation area of the gate apparatus 10 including the reading unit108. That is, the fixed camera 20 captures an image on the entrance sideto the installation area of the reading unit 108. Thereby, the fixedcamera 20 can capture a visitor V in an area in front of the gateapparatus 10 that is the entrance side to the installation area of thereading unit 108. Therefore, the fixed camera 20 can capture an imageincluding an authentication subject.

The fixed camera 20 is a digital video camera, for example, and is ableto capture a moving image at a predetermined framerate to continuouslyacquire a plurality of images at a predetermined cycle synchronized withthe framerate. For example, the fixed camera 20 is able to capture amoving image at 15 fps and continuously acquire images of 15 frame persecond.

Note that the fixed camera 20 may be a digital still camera. In thiscase, the fixed camera 20 can be configured to continuously capturestatic images at a predetermined capturing interval and continuouslyacquire a plurality of images at a predetermined cycle.

Further, the fixed camera 20 may be a visible light camera or aninfrared camera. When the fixed camera 20 is an infrared camera, aninfrared lighting apparatus 506 that emits an infrared may be providedto the roof 502 of the entrance gate 50 in addition to the normallighting apparatus 504 that emits an illumination light including avisible light. By using an infrared camera as the fixed camera 20 underthe infrared lighting apparatus 506, it is possible to perform facematching based on a face image captured by the fixed camera 20 whilereducing the influence by the brightness nearby.

Further, the fixed camera 20 is installed in a vertical orientation soas to capture a vertically long image. This enables the fixed camera 20to capture an image including a visitor's face of a wide range ofheights from a short visitor to a tall visitor. Specifically, the fixedcamera 20 can capture an image including a face of a visitor whoseheight ranges from 99 cm to 220 cm, for example. Note that the fixedcamera 20 is not necessarily required to be installed vertically but maybe installed horizontally so as to capture a horizontally long image.

Faces of a plurality of visitors in an area in front of the gateapparatus 10 may be included in an image captured by the fixed camera20.

The fixed camera 20 transmits image data of a plurality of imagesacquired at a predetermined cycle as described above to the facematching apparatus 30 in synchronization with the cycle.

The face matching apparatus 30 has a face matching control unit 302, astorage unit 304, and a display unit 306.

The face matching control unit 302 performs face matching based on aface image captured by the fixed camera 20. The face matching controlunit 302 includes an image data acquisition unit 308, a face detectionunit 310, a face feature amount extraction unit 312, and a face matchingunit 314.

The image data acquisition unit 308 sequentially acquires image data ofimages transmitted from the fixed camera 20 at a predetermined cycle.Note that the image data acquisition unit 308 can perform imageprocessing such as a correction process on the acquired image data.

The face detection unit 310 performs face detection on respective imagesof image data sequentially acquired from the image data acquisition unit308. Thereby, the face detection unit 310 detects a face image of avisitor in an area in front of the gate apparatus 10 as a detected faceimage out of images of image data sequentially acquired by the imagedata acquisition unit 308. As an algorithm used by the face detectionunit 310 for face detection, without being limited in particular,various algorithms may be used.

FIG. 3 illustrates an example of one frame of image captured by thefixed camera 20. As illustrated in FIG. 3 , a single visitor V1 in anarea in front of the gate apparatus 10 is captured in one frame of imageI. The face detection unit 310 detects a face image of the visitor V1 asillustrated by a rectangular detection frame 316 in FIG. 3 from theimage I captured by the fixed camera 20 in such a way.

Note that a plurality of visitors in an area in front of the gateapparatus 10 may be captured in one frame of image captured by the fixedcamera 20. Further, the same person may be captured in different framesof images captured by the fixed camera 20. These cases will be describedlater.

The face feature amount extraction unit 312 extracts a face featureamount that is a feature amount of a face image for respective faceimages detected by the face detection unit 310. Note that a face imagedetected by the face detection unit 310 may be referred to as a detectedface image below. The face feature amount is a vector amount andobtained by combining scaler amount components expressing the feature ofa face image. As a component of a feature amount, without being limitedin particular, various types thereof may be used. For example, as acomponent of a feature amount, a positional relationship such as adistance or an angle between feature points that are set at the centeror the end of an organ of a face, such as an eye, a nose, a mouth, orthe like, a curvature of the outline of a face, a color distribution ora shade and light value of the surface of a face, or the like can beused. The number of components of the feature amount may be set asappropriate in accordance with required matching accuracy, a processingspeed, or the like without being limited in particular.

Further, the face feature amount extraction unit 312 temporarily storesface image data that is image data of the detected face image togetherwith the face feature amount extracted from the detected face image inthe storage unit 304 in association with each other. Furthermore, theface feature amount extraction unit 312 temporarily stores a detectionnumber that is a number identifying the image data and the capturingtime when the detected face image is captured in the storage unit 304 inassociation with each other for respective detected face images,together with face image data and the face feature amount thereof.

A relational database is configured in the storage unit 304. In therelational database of the storage unit 304, the face feature amountextracted by the face feature amount extraction unit 312 as describedabove is temporarily stored in association with a detection number, acapturing time, and face image data. Such mutually associated data ismanaged by a Relational Database Management System (RDBMS). As an RDBMS,without being limited in particular, Microsoft (registered trademark)SQL Server is used, for example.

FIG. 4 is a schematic diagram illustrating an example of various datafor detected face images temporarily stored in the storage unit 304 bythe face feature amount extraction unit 312. As illustrated in FIG. 4 ,a detection number, a capturing time, face image data, and a facefeature amount are associated with each other and temporarily stored inthe storage unit 304 for respective detected face images.

In the storage unit 304, the face feature amount and data relatedthereto are stored for only a certain period from the capturing time foreach detected face image. The face feature amount and data relatedthereto of a detected face image remaining after a certain time haselapsed from the capturing time are sequentially deleted from therelational database of the storage unit 304. For example, the facefeature amount and data related thereto of the detected face imagecaptured by the fixed camera 20 within the immediate past three minutesare temporarily stored in the storage unit 304.

When there is ticket presentation of an annual pass to the reading unit108 of the gate apparatus 10, the face matching unit 314 performsidentity verification by face matching for the visitor who performsticket presentation of an annual pass at the reading unit 108.

ID information read by the reading unit 108 from the annual pass on theticket presentation is transmitted to the face matching unit 314. Theface matching unit 314 acquires the transmitted ID information andacquires, online, a face feature amount of a registered face image whichis registered in association with the ID information via the network 60from the datacenter server 40 described later. A person of theregistered face image acquired by the face matching unit 314 as above isa valid user who can validly use the annual pass on the ticketpresentation. A valid user of an annual pass is a purchaser who haspurchased the annual pass, for example.

Further, the face matching unit 314 refers to the relational database ofthe storage unit 304 and acquires, offline, face feature amounts of Ndetected face images associated with the capturing time included in apredetermined period before ticket presentation that is beforeacquisition of ID information. That is, the face matching unit 314acquires face feature amounts of N detected face images captured by thefixed camera 20 before the reading unit 108 reads ID information from anannual pass. Note that N is typically an integer greater than one, and aplurality of detected face images are acquired by the face matching unit314. However, there may be a case where N is one and a single detectedface image is acquired by the face matching unit 314. A predeterminedperiod before ticket presentation is performed for acquiring a detectedface image may be a period immediately before the ticket presentation,and the length thereof may be set as appropriate in accordance withrequired matching accuracy, a processing speed, or the like. Forexample, a predetermined period before ticket presentation for acquiringa detected face image can be set to several seconds immediately beforethe ticket presentation.

The face matching unit 314 performs a matching process that sequentiallymatches respective face feature amounts of N detected face images, whichhave been captured before ticket presentation of an annual pass, againsta face feature amount of a registered face image. The matching processhere is referred to as N:1 matching because matching of the maximum Ndetected face images against one registered face image is performed. Asdiscussed above, the face matching unit 314 matches detected faceimages, which have been detected by the face detection unit 310 fromimages captured before acquisition of ID information, against aregistered face image corresponding to the acquired ID information.

The face matching unit 314 calculates a matching score in accordancewith a similarity between a face feature amount of a detected face imageand a face feature amount of a registered face image in N:1 matching.The matching score is a larger value for a higher similarity between theface feature amount of a detected face image and the face feature amountof a registered face image. As a result of matching for a certaindetected face image, the face matching unit 314 determines that thematching is unmatched if the matching score is less than a predeterminedthreshold and performs matching of the face feature amount of the nextdetected face image with the face feature amount of the registered faceimage. On the other hand, as a result of matching for a certain detectedface image, the face matching unit 314 determines that the matching ismatched if the matching score is greater than or equal to thepredetermined threshold and completes the matching process.

The order of performing matching of face feature amounts against aregistered face image for N detected face images is not limited inparticular. For example, for N detected face images, matching of facefeature amounts against a registered face image may be performed inascending order or descending order of capturing time or at random.Further, a priority may be determined for each of the N detected faceimages, and the order of performing matching of face feature amountsagainst a registered face image may be determined based on the priority.Note that the case where the order of performing matching of facefeature amounts against a registered face image is determined based onthe priority will be described in a second embodiment.

FIG. 5 is a schematic diagram illustrating an example of a process ofN:1 matching performed by the face matching unit 314 in the presentexample embodiment. As illustrated in FIG. 5 , face feature amounts of Ndetected face images captured in a predetermined period before ticketpresentation are sequentially matched against a face feature amount of aregistered face image registered in association with ID information ofan annual pass on the ticket presentation. The order of performingmatching of face feature amounts against a registered face image for Ndetected face images may be ascending order of capturing time, forexample.

If the matching performed by the face matching unit 314 is matched, thismeans that a valid user of an annual pass on the ticket presentation hasbeen included in visitors in front of the gate apparatus 10 before theticket presentation. Thus, it can be estimated that a valid user of anannual pass performs the ticket presentation of the annual pass.Therefore, in this case, identity verification by face matching issuccessful.

On the other hand, if all the matching performed by the face matchingunit 314 is unmatched, no valid user of an annual pass on the ticketpresentation has been included in visitors in front of the gateapparatus 10 before the ticket presentation. Therefore, in this case,identity verification by face matching is failed.

A matching result or the like by the face matching unit 314 can bedisplayed on the display unit 306. The staff of the facility can confirma matching result or the like by viewing the display on the display unit306.

The face matching unit 314 transmits a matching result signal that is asignal indicating the matching result described above to the gateapparatus 10. Specifically, the face matching unit 314 transmits, to thegate apparatus 10, a matching-matched signal that is a signal indicatingthat the matching by the face matching unit 314 is matched or amatching-unmatched signal that is a signal indicating that all thematching performed by the face matching unit 314 is unmatched.

The face matching apparatus 30 described above is formed of a computerapparatus, for example. An example of a hardware configuration of theface matching apparatus 30 will be described by using FIG. 6 . Note thatthe face matching apparatus 30 may be formed of a single device or maybe formed of two or more physically separated devices in wiredconnection or wireless connection.

As illustrated in FIG. 6 , the face matching apparatus 30 has a centralprocessing unit (CPU) 3002, a read only memory (ROM) 3004, a randomaccess memory (RAM) 3006, and a hard disk drive (HDD) 3008. Further, theface matching apparatus 30 has a communication interface (I/F) 3010.Further, the face matching apparatus 30 has a display controller 3012and the display 3014. Furthermore, the face matching apparatus 30 has aninput device 3016. The CPU 3002, the ROM 3004, the RAM 3006, the HDD3008, the communication I/F 3010, the display controller 3012, and theinput device 3016 are connected to a common bus line 3018.

The CPU 3002 controls the entire operation of the face matchingapparatus 30. Further, the CPU 3002 executes a program that implementsthe function of each unit of the image data acquisition unit 308, theface detection unit 310, the face feature amount extraction unit 312,and the face matching unit 314 in the face matching control unit 302described above. The CPU 3002 loads a program stored in the HDD 3008 orthe like to the RAM 3006 to implement the function of each unit of theface matching control unit 302.

The ROM 3004 stores a program such as a boot program therein. The RAM3006 is used as a working area when the CPU 3002 executes a program.Further, the program executed by the CPU 3002 is stored in the HDD 3008.

Further, the HDD 3008 is a storage device that implements the functionof the storage unit 304 described above. Note that a storage device thatimplements the function of the storage unit 304 is not limited to theHDD 3008. Various storage devices can be used for implementing thefunction of the storage unit 304.

The communication I/F 3010 is connected to the network 60. Thecommunication I/F 3010 controls data communication with the datacenterserver 40 connected to the network 60.

The display controller 3012 is connected to the display 3014 thatfunctions as the display unit 306. The display controller 3012 causes amatching result from the face matching unit 314 to be displayed on thedisplay 3014.

The input device 3016 may be a keyboard, a mouse, or the like, forexample. Further, the input device 3016 may be a touch panel embedded inthe display 3014. The staff of the facility can perform setting of theface matching apparatus 30 or input an instruction of execution of aprocess via the input device 3016.

Note that the hardware configuration of the face matching apparatus 30is not limited to the configuration described above, but may be variousconfigurations.

The gate control unit 112 of the gate apparatus 10 controls opening andclosing of the gate 106 based on a matching result signal transmittedfrom the face matching unit 314. That is, the gate control unit 112opens the gate 106 when a matching-matched signal is transmitted fromthe face matching unit 314. Thereby, a visitor performing ticketpresentation is allowed to walk through the path 114 of the gateapparatus 10 to enter the inside of the facility as a person who hasbeen successful in identity verification. The gate control unit 112causes the gate 106 to be closed after the visitor walked through thepath 114.

On the other hand, the gate control unit 112 maintains a closed state ofthe gate 106 when a matching-unmatched signal is transmitted from theface matching unit 314. At this time, the gate control unit 112 cansound an alert sound of a not-shown alarm provided to the gate apparatus10, turn on an alert light, or the like to output a warning indicatingthat all the matching results are unmatched.

The datacenter server 40 has a control unit 402 and a storage unit 404.

The control unit 402 controls the operation of the datacenter server 40.

The storage unit 404 accumulates registered face images and face featureamounts thereof that are registered in association with ID informationof issued annual passes.

The control unit 402 provides, to the face matching unit 314, a facefeature amount of a registered face image registered in association withID information of an annual pass on ticket presentation in response to arequest from the face matching unit 314.

A registered face image can be uploaded on the web server 70 from apurchaser's terminal when an annual pass is purchased at a web storeprovided by the web server 70. The registered face image uploaded to theweb server 70 is transmitted from the web server 70 to the datacenterserver 40. In the datacenter server 40 to which the registered faceimage has been transmitted, the control unit 402 accumulates thetransmitted registered face image in the storage unit 404.

Further, a purchaser who purchased an annual pass at a ticket booth cancapture his/her registration face image by the hand camera 110 of thegate apparatus 10 when first visiting the facility. The registered faceimage captured by the hand camera 110 is transmitted to the datacenterserver 40 via the network 60 by the face matching apparatus 30 andaccumulated in the storage unit 404 thereof.

Further, a purchaser who purchased an annual pass at a ticket booth maysoon capture his/her registration face image by a hand camera (notshown) of the ticket booth. The registered face image captured by thehand camera of the thicket booth is transmitted to the datacenter server40 via the network 60 by the ticket booth terminal 80 and accumulated inthe storage unit 404 thereof.

For a registered face image accumulated in the storage unit 404 of thedatacenter server 40 as described above, the same face feature amount asthe face feature amount extracted by the face feature amount extractionunit 312 of the face matching apparatus 30 is extracted. Extraction of aface feature amount is performed by the control unit 402 that functionsas a face feature amount extraction unit. The extracted face featureamount is accumulated in the storage unit 404 by the control unit 402 inassociation with ID information of the annual pass with which theregistered face image is associated.

A relational database is configured in the storage unit 404. In therelational database of the storage unit 404, face feature amounts ofregistered face images are stored in association with ID information ofannual passes and face image data of the registered face images, asdescribed above. Such mutually associated data is managed by an RDBMS.As an RDBMS, without being limited in particular, Microsoft (registeredtrademark) SQL Server is used, for example.

Note that, in the relational database of the storage unit 404, inaddition to the above, pieces of information such as names, contactaddresses, or the like of purchasers of annual passes who are validusers of the annual passes are stored in association with ID informationof the annual passes, for example.

The datacenter server 40 described above is formed of a computerapparatus, for example. An example of a hardware configuration of thedatacenter server 40 will be described by using FIG. 7 . Note that thedatacenter server 40 may be formed of a single device or may be formedof two or more physically separated devices by wired connection orwireless connection.

As illustrated in FIG. 7 , the datacenter server 40 has a CPU 4002, aROM 4004, a RAM 4006, and an HDD 4008. Further, the datacenter server 40has a communication I/F 4010. The CPU 4002, the ROM 4004, the RAM 4006,the HDD 4008, and the communication I/F 4010 are connected to a commonbus line 4012.

The CPU 4002 controls the entire operation of the datacenter server 40.Further, the CPU 4002 executes a program that implements the function ofthe control unit 402 described above. The CPU 4002 loads a programstored in the HDD 4008 or the like to the RAM 4006 to implement thefunction of the control unit 402.

The ROM 4004 stores a program such as a boot program therein. The RAM4006 is used as a working area when the CPU 4002 executes a program.Further, the program executed by the CPU 4002 is stored in the HDD 4008.

Further, the HDD 4008 is a storage device that implements the functionof the storage unit 404 described above. Note that a storage device thatimplements the function of the storage unit 404 is not limited to theHDD 4008. Various storage devices can be used for implementing thefunction of the storage unit 404.

The communication I/F 4010 is connected to the network 60. Thecommunication I/F 4010 controls data communication with the facematching apparatus 30 connected to the network 60.

Note that the hardware configuration of the datacenter server 40 is notlimited to the configuration described above, but may be variousconfigurations.

As described above, the face recognition system 1 according to thepresent example embodiment matches a detected face image captured by thefixed camera 20 before ticket presentation, in which the reading unit108 reads ID information from an annual pass is performed, against aregistered face image registered in association with ID information ofthe annual pass on the ticket presentation. That is, in the facerecognition system 1 according to the present example embodiment, adetected face image that is an image of a matching subject to be matchedagainst a registered face image is acquired in advance before ticketpresentation of an annual pass.

Thus, according to the present example embodiment, after a visitorperforms ticket presentation of an annual pass, it is not necessary forthe staff of the facility to capture a face image of the visitor as animage of a matching subject to be matched against a registered faceimage. Further, a visitor neither needs to concern about capturing ofthe face image thereof nor needs to perform a special move such aspositioning of the face thereof for the capturing. Therefore, accordingto the present example embodiment, face matching can be made smoothly ina short time.

Next, a face recognition method according to the present exampleembodiment using the face recognition system 1 according to the abovepresent example embodiment will be further described by using FIG. 8 toFIG. 10 .

First, the entire flow of the face recognition method according to thepresent example embodiment will be described by using FIG. 8 . FIG. 8 isa sequence diagram illustrating the face recognition method according tothe present example embodiment.

The fixed camera 20 captures an area in front of the gate apparatus 10that is the entrance side to the installation area of the reading unit108 and continuously acquires a plurality of images at a predeterminedcycle (step S102). Further, the fixed camera 20 transmits image data ofthe plurality of images acquired at a predetermined cycle to the facematching apparatus 30 in synchronization with the cycle (step S104).

In the face matching apparatus 30 to which the image data has beentransmitted, the image data acquisition unit 308 sequentially acquiresimage data transmitted from the fixed camera 20. The face detection unit310 performs face detection for each image and detects a face image as adetected face image (step S106). The face feature amount extraction unit312 extracts a face feature amount for each detected face image fortemporary storage (step S108).

In the fixed camera 20 and the face matching apparatus 30, steps 5102 to5108 described above are repeatedly performed.

On the other hand, when there is ticket presentation of an annual passat the gate apparatus 10, the reading unit 108 reads ID information ofthe annual pass on the ticket presentation (step S110). Subsequently,the gate control unit 112 transmits ID information read by the readingunit 108 to the face matching apparatus 30 (step S112).

In the face matching apparatus 30 to which the ID information has beentransmitted, the face matching unit 314 transmits the ID information tothe datacenter server 40 via the network 60 (step S114) and requests aface feature amount of a registered face image registered in associationwith the ID information. Thereby, the face matching unit 314 acquires aface feature amount of a registered face image registered in associationwith the ID information online from the datacenter server 40 via thenetwork 60 (step S116).

Further, the face matching unit 314 refers to a relational database ofthe storage unit 304 thereof and acquires, offline, face feature amountsof N detected face images associated with the capturing time included ina predetermined period before the ticket presentation (step S118). Thatis, the face matching unit 314 acquires face feature amounts of Ndetected face images captured before the reading unit 108 reads IDinformation from the annual pass.

Note that any one of the above step S116 and step S118 may be performedearlier or both of the above step S116 and step 5118 may be performed atthe same time.

Next, the face matching unit 314 performs N:1 matching based on the facefeature amounts of the acquired detected face images and the facefeature amount of the registered face image (step S120).

The face matching unit 314 that has performed N:1 matching transmits amatching result signal indicating a matching result to the gateapparatus 10 (step S122).

In the gate apparatus 10 to which the matching result signal has beentransmitted, the gate control unit 112 controls opening and closing ofthe gate 106 based on the matching result signal (step S124).

Every time ticket presentation of an annual pass is performed at thereading unit 108 of the gate apparatus 10, steps S110 to S124 arerepeatedly performed.

Next, details of the process in the face recognition method according tothe present example embodiment will be described by using FIG. 9 andFIG. 10 . FIG. 9 and FIG. 10 are flowcharts illustrating the facerecognition method according to the present example embodiment,respectively.

First, a process from capturing by the fixed camera 20 to temporarystorage of a face feature amount of a detected face image will bedescribed in detail by using FIG. 9 .

The image data of images captured by the fixed camera 20 is periodicallytransmitted to the face matching apparatus 30. The image dataacquisition unit 308 determines whether or not it is the timing oftransmission of image data from the fixed camera 20 (step S202). If itis not the timing of transmission of image data (step S202, NO), theimage data acquisition unit 308 stands by for an arrival of the timing.

If it is the timing of transmission of image data from the fixed camera20 (step S202, YES), the image data acquisition unit 308 acquires imagedata transmitted from the fixed camera 20 (step S204). Before the nextstep S206, the image data acquisition unit 308 can perform imageprocessing such as a correction process on the acquired image data.

Next, the face detection unit 310 performs face detection on an image ofimage data acquired by the image data acquisition unit 308 and, from theimage, detects a face image of a visitor in the area in front of thegate apparatus 10 (step S206). If no face image is detected (step S208,NO), the process returns to step S202 and stands by for transmission ofnext image data from the fixed camera 20.

If a face image is detected by the face detection unit 310 (step S208,YES), the face feature amount extraction unit 312 extracts a facefeature amount for a detected face image that is the detected face image(step S210).

Further, the face feature amount extraction unit 312 temporarily storesthe face image data of the detected face image together with the facefeature amount extracted from the detected face image in the storageunit 304 in association with each other (step S212). At this time, theface feature amount extraction unit 312 temporarily stores a detectionnumber that is a number identifying the image data and the capturingtime when the detected face image is captured in the storage unit 304 inassociation with each other for the detected face image, together withface image data and the face feature amount thereof.

The process illustrated in FIG. 9 described above is repeatedlyperformed every time the timing of transmission of image data from thefixed camera 20 arrives during the operation of the face recognitionsystem 1 according to the present example embodiment.

Next, a process from ticket presentation of an annual pass to gatecontrol will be described by using FIG. 10 .

In the gate apparatus 10, the reading unit 108 stands by until ticketpresentation of an annual pass is performed (step S302, NO). Duringstandby for ticket presentation, the gate 106 is closed.

In response to ticket presentation of an annual pass by a visitor at thereading unit 108 (step S302, YES), the reading unit 108 reads IDinformation recorded in the annual pass from the annual pass (stepS306).

Subsequently, the gate control unit 112 transmits the ID informationread by the reading unit 108 to the face matching apparatus 30 (stepS306).

In the face matching apparatus 30, the face matching unit 314 transmits,to the datacenter server 40 via the network 60, the ID informationtransmitted from the gate control unit 112 and requests for a facefeature amount of a registered face image. Thereby, the face matchingunit 314 acquires, online, from the datacenter server 40 via the network60, a face feature amount of a registered face image registered inassociation with the transmitted ID information (step S308).

Further, the face matching unit 314 refers to a relational database ofthe storage unit 304 to acquire, offline, face feature amounts of Ndetected face images associated with the capturing time included in apredetermined period before ticket presentation of an annual pass (stepS310).

The face matching unit 314 performs N:1 matching that sequentiallymatches respective face feature amounts of the acquired N detected faceimages against the face feature amount of the acquired registered faceimage (step S312). If the matching score is less than a predeterminedthreshold as a result of matching for a certain detected face image, theface matching unit 314 determines that the matching is unmatched andperforms matching of the face feature amount of the next detected faceimage with the face feature amount of the registered face image. On theother hand, if the matching score is greater than or equal to thepredetermined threshold as a result of matching for the certain detectedface image, the face matching unit 314 determines that the matching ismatched and completes the matching process.

If the matching performed by the face matching unit 314 is matched (stepS314, YES), the face matching unit 314 transmits a matching-matchedsignal indicating that the matching performed by the face matching unit314 is matched to the gate apparatus 10 (step S316).

In the gate apparatus 10, in response to transmission of amatching-matched signal from the face matching unit 314, the gatecontrol unit 112 opens the gate 106 (step S318). Thereby, the visitorperforming ticket presentation is allowed to walk through 114 of thegate apparatus 10 and enter the inside of the facility as a person whois successful in identity verification.

The gate control unit 112 closes the gate 106 after the visitor haspassed through the path 114 (step S320).

On the other hand, if all the matching performed by the face matchingunit 314 is unmatched (step S314, NO), the face matching unit 314transmits a matching-unmatched signal indicating that all the matchingperformed by the face matching unit 314 is unmatched to the gateapparatus 10 (step S322).

In the gate apparatus 10, in response to transmission of amatching-unmatched signal from the face matching unit 314, the gatecontrol unit 112 maintains the closed state of the gate 106 (step S324).At this time, the gate control unit 112 outputs a warning by sounding analert sound of a not-shown alert provided to the gate apparatus 10,turning on an alert light, or the like (step S326). As a result, thevisitor performing ticket presentation is unable to enter the inside ofthe facility at this stage as a person who is failed in identityverification. In this case, an action such as identity verificationagain performed by the staff of the facility is made, for example.

The process illustrated in FIG. 10 described above is repeatedlyperformed every time ticket presentation is made at the reading unit 108of the gate apparatus 10 during the face recognition method according tothe present example embodiment being performed.

As discussed above, according to the present example embodiment, a facefeature amount of a detected face image detected from an image capturedby the fixed camera 20 before ticket presentation of an annual pass isperformed is matched against a face feature amount of a registered faceimage registered in association with ID information of the annual passon the ticket presentation. Therefore, according to the present exampleembodiment, face matching can be performed smoothly in a short time.

Note that, while the case where a single visitor is captured in oneframe of image captured by the fixed camera 20 as illustrated in FIG. 3has been described above as an example, a plurality of visitors may becaptured in one frame of image captured by the fixed camera 20.

FIG. 11 illustrates another example of one frame of image captured bythe fixed camera 20. As illustrated in FIG. 11 , in one frame of imageI, three visitors Vp, Vq, and Vr are captured in an area in front of thegate apparatus 10. In this way, a plurality of visitors may be capturedin one frame of image I captured by the fixed camera 20. In such a case,the face detection unit 310 detects respective face images of theplurality of visitors Vp, Vq, and Vr from the image I captured by thefixed camera 20 as indicated by detection frames 316 p, 316 q, and 316 rin FIG. 11 . The plurality of detected face images detected from oneframe of image in such a way are processed as an image of a matchingsubject to be matched against the registered face image, respectively,in the same manner as described above.

Second Example Embodiment

A face recognition system and a face recognition method according to asecond example embodiment of the present invention will be described byusing FIG. 12 and FIG. 13 . FIG. 12 is a block diagram illustrating afunctional configuration of a face recognition system according to thepresent example embodiment. FIG. 13 is a schematic diagram illustratingan example of a process of N:1 matching in the face recognition systemaccording to the present example embodiment. Note that componentssimilar to those in the face recognition system and the face recognitionmethod according to the first example embodiment described above will belabeled with the same reference numeral, and the description thereofwill be omitted or simplified.

The basic configuration of the face recognition system according to thepresent example embodiment is substantially the same as theconfiguration of the face recognition system according to the firstexample embodiment. The face recognition system according to the presentexample embodiment is different from the face recognition systemaccording to the first example embodiment in that the face matchingapparatus 30 further has a priority calculation unit that calculates thepriority in performing N:1 matching for a detected face image.

As illustrated in FIG. 12 , in a face recognition system 2 according tothe present example embodiment, the face matching control unit 302 ofthe face matching apparatus 30 further has a priority calculation unit318.

The priority calculation unit 318 calculates a priority used fordetermining the order of performing matching of a face feature amountagainst a registered face image for respective detected face imagesdetected by the face detection unit 310. The priority calculation unit318 can calculate a priority based on various factors.

As a factor by which the priority calculation unit 318 calculates apriority, a positional relationship of detected face images in an imagecaptured by the fixed camera 20 is exemplified. A person of a detectedface image located closer to the reading unit 108 of the gate apparatus10 in an image captured by the fixed camera 20 is more likely to performticket presentation of an annual pass at the reading unit 108.Therefore, a higher priority can be set for a detected face imagelocated closer to the reading unit 108 in an image captured by the fixedcamera 20.

Further, as a factor by which the priority calculation unit 318calculates a priority, a facial likeness score that is an evaluationvalue evaluating a facial likeness of a detected face image can beexemplified. For example, a facial likeness score having a higher valueindicates that a detected face image is more likely to be a face.Therefore, a higher priority can be set for a detected face image havinga higher facial likeness score.

Further, as a factor by which the priority calculation unit 318calculates the priority, a quality of a detected face image isexemplified. A higher quality of a detected face image enables a moreaccurate matching. Therefore, a higher priority can be set for adetected face image having a higher quality.

The priority calculation unit 318 can calculates the priority based onany of the factors described above or a combination of these factors.For example, the priority value may be a smaller value for a higherpriority.

The priority calculation unit 318 stores the calculated priorities forrespective detected face images in the relational database configured inthe storage unit 304 in association with a detection number, a capturingtime, and face image data.

Note that, also in the present example embodiment, the face matchingapparatus 30 can have the same hardware configuration as the hardwareconfiguration illustrated in FIG. 6 . In this case, the CPU 3002executes a program implementing the function of the priority calculationunit 318.

FIG. 13 is a schematic diagram illustrating an example of a process ofN:1 matching performed by the face matching unit 314 in the presentexample embodiment. As illustrated in FIG. 13 , sequentially indescending order of priority, the face matching unit 314 matches theface feature amounts of N detected face images, which have been capturedin a predetermined period before ticket presentation of an annual pass,against a face feature amount of a registered face image registered inassociation with ID information of the annual pass on the ticketpresentation.

As discussed above, in the present example embodiment, since the orderof performing matching of a face feature amount against a registeredface image for N detected face images is descending order of priority,the N:1 matching can be efficiently performed. Therefore, according tothe present example embodiment, face matching can be performed smoothlyin a short time.

Third Example Embodiment

A face recognition system and a face recognition method according to athird example embodiment of the present invention will be described byusing FIG. 14A, FIG. 14B, and FIG. 15 . FIG. 14A and FIG. 14B areschematic diagrams illustrating examples of different images captured bya fixed camera in the face recognition system according to the presentexample embodiment. FIG. 15 is a block diagram illustrating a functionalconfiguration of a face recognition system according to the presentexample embodiment. Note that components similar to those in the facerecognition system and the face recognition method according to thefirst and second example embodiments described above will be labeledwith the same reference numeral, and the description thereof will beomitted or simplified.

The basic configuration of the face recognition system according to thepresent example embodiment is substantially the same as the facerecognition system according to the configuration of the first exampleembodiment. The face recognition system according to the present exampleembodiment is different from the face recognition system according tothe first example embodiment in that the face matching apparatus 30further has an identical-person processing unit that performs a processwhen visitors of an identical person are captured in different frames ofimages captured by the fixed camera 20.

When an area in front of the gate apparatus 10 is captured by the fixedcamera 20, visitors of an identical person may be captured in differentframes of images.

FIG. 14A and FIG. 14B illustrate different frames of images Im and Insuccessively captured by the fixed camera 20, respectively. The image Inis a later frame of image than the image Im. In the image Im, visitorsVs, Vt, and Vu are captured. On the other hand, also in the image Inlater than the image Im, the visitor Vs who was captured in the image Imis captured.

In the present example embodiment, when visitors of an identical personare captured in a different frame of images in such a way, the detectedface images are classified on an identical person basis. The prioritybased on the quality or the like is then determined for the detectedface image of the identical person, and matching of a face featureamount against a registered face image is performed in descending orderof priority. This allows more efficient N:1 matching to be performed.

As illustrated in FIG. 15 , in a face recognition system 3 according tothe present example embodiment, the face matching control unit 302 ofthe face matching apparatus 30 further has an identical-personprocessing unit 320.

The identical-person processing unit 320 classifies detected face imagesdetected by the face detection unit 310 into respective identicalpersons. Furthermore, the identical-person processing unit 320calculates a priority for determining the order of performing matchingof a face feature amount against the registered face image for aplurality of detected face images classified into identical persons. Theidentical-person processing unit 320 can calculate the priority based ona positional relationship of detected face images, a facial likenessscore, a quality, or the like in a similar manner to the prioritycalculation unit 318 according to the second example embodiment.

Note that, also in the present example embodiment, the face matchingapparatus 30 can have the same hardware configuration as the hardwareconfiguration illustrated in FIG. 6 . In this case, the CPU 3002executes a program implementing the function of the identical-personprocessing unit 320.

The face matching unit 314 performs matching of face feature amountsagainst a registered face images in descending order of priority for aplurality of detected face images classified into an identical person inN:1 matching.

As discussed above, according to the present example embodiment, in N:1matching, since matching of face feature amounts against a registeredface image is performed in descending order of priority for detectedface images classified into an identical person, the N:1 matching can beperformed efficiently. Therefore, according to the present exampleembodiment, face matching can be performed more smoothly in a shortertime.

Note that, while the identical-person processing unit 320 is furtherprovided in addition to the configuration of the face recognition systemaccording to the first example embodiment above has been describedabove, the example embodiment is not limited thereto. Theidentical-person processing unit 320 may be further provided in additionto the configuration of the face recognition system according to thesecond example embodiment.

Fourth Example Embodiment

A face recognition system and a face recognition method according to afourth example embodiment of the present invention will be described byusing FIG. 16 , FIG. 17A, FIG. 17B, and FIG. 17C. FIG. 16 is a flowchartillustrating automatic update of a registered face image in a facerecognition system according to the present example embodiment. FIG.17A, FIG. 17B, and FIG. 17C are schematic diagrams illustrating variousdata in automatic update of a registered face image in the facerecognition system according to the present example embodiment. Notethat components similar to those in the face recognition system and theface recognition method according to the first to third exampleembodiments described above will be labeled with the same referencenumeral, and the description thereof will be omitted or simplified.

The basic configuration of the face recognition system according to thepresent example embodiment is substantially the same as theconfiguration of the face recognition system according to the firstexample embodiment. When the matching performed by the face matchingunit 314 of the face matching apparatus 30 is matched, the facerecognition system according to the present example embodiment updates aregistered face image stored in the datacenter server 40 based on thematching result.

An update process of a registered face image in a face recognitionsystem according to the present example embodiment will be describedbelow by using FIG. 16 , FIG. 17A, FIG. 17B, and FIG. 17C.

In the face matching apparatus 30, if matching is matched as a result ofN:1 matching, the face matching unit 314 stores the matching result(step S402). The face matching unit 314 can store the matching result inthe storage unit 304 or can store the matching result in a storage unitprovided separately from the storage unit 304.

FIG. 17A illustrates an example of a matching result stored by the facematching unit 314. As illustrated in FIG. 17A, ID information of anannual pass on ticket presentation, image data of a detected face imagematched to matching against a registered face image, a face featureamount of the detected face image, and a matching score are stored inassociation with each other. The matching result is stored for each IDinformation of an annual pass on ticket presentation. Note that the facematching unit 314 can further store a facial likeness score of adetected face image in which a matching is matched as well as otherinformation in association together, for example.

Next, the face matching unit 314 transmits the stored matching result tothe datacenter server 40 regularly or irregularly (step S404). Forexample, the face matching unit 314 can transmit a matching result tothe datacenter server 40 on a daily basis.

In the datacenter server 40 to which a matching result is transmitted,the control unit 402 stores the transmitted matching result (step S406).The control unit 402 can store the matching result in the storage unit404 or can store a storage unit provided separately from the storageunit 404.

Next, the control unit 402 regularly or irregularly processes the storedmatching result. Specifically, the control unit 402 stores the detectedface image associated with ID information in the storage unit 404 bynewly adding it as a registered face image associated with the IDinformation (step S408). Further, the face feature amount of the newlyadded registered face image is stored in the storage unit 404 inassociation with the ID information. In such a way, for the same IDinformation, a plurality of registered face images and the face featureamounts thereof are stored and registered in association with eachother. As discussed above, the control unit 402 functions as an updateunit that additionally registers the detected face image in whichmatching is matched to the registered face image as a new registeredface image and additionally updates the registered face image.

FIG. 17B illustrates a plurality of registered face images registered inassociation with the same ID information and the face feature amountsthereof. As illustrated in FIG. 17B, the control unit 402 additionallystores a detected face image in which matching is matched as a newregistered face image together with the face feature amount in additionto existing registered face images.

Next, the control unit 402 updates the priority of a plurality ofregistered face images stored in association with the same IDinformation (step S410). The control unit 402 can calculate a prioritybased on a facial likeness score or the like in a similar manner to thepriority calculation unit 318 according to the second example embodimentand update the priority of the plurality of registered face images.

FIG. 17C illustrates a case where a priority is calculated for aplurality of registered face images illustrated in FIG. 17B.

The plurality of registered face images whose priorities have beenupdated in such a way are subjected to the N:1 matching performed by theface matching unit 314 of the face matching apparatus 30 in descendingorder of priority.

The process illustrated in FIG. 16 described above is performedrepeatedly during the face recognition system according to the presentexample embodiment being operated.

As described above, according to the present example embodiment, sincethe registered face images stored in the datacenter server 40 areupdated based on a matching result from the face matching unit 314 ofthe face matching apparatus 30, the accuracy of face matching can bemaintained to be high.

Note that, while the case where the registered face images are updatedin the face recognition system according to the first example embodimenthas been described above, the example embodiment is not limited thereto.The registered face images can be updated in the same manner as above inthe face recognition system according to the second and third exampleembodiment.

Fifth Embodiment

A face recognition system and a face recognition method according to afifth example embodiment of the present invention will be described byusing FIG. 18 . FIG. 18 is a flowchart illustrating the face recognitionmethod according to the present example embodiment. Note that componentssimilar to those in the face recognition system and the face recognitionmethod according to the first to fourth example embodiments describedabove will be labeled with the same reference numeral, and thedescription thereof will be omitted or simplified.

In the present example embodiment, when identity verification by facematching based on an image captured by the fixed camera 20 is failed, aface image of a visitor is captured by the hand camera 110 to performface matching in the first example embodiment. The face recognitionmethod according to the present example embodiment will be describedbelow by using FIG. 18 .

Also in the present example embodiment, identity verification by facematching is performed based on an image captured by the fixed camera 20in the same manner as in the first example embodiment. As illustrated inFIG. 18 , since steps S302 to S314, steps S316 to S320 when matching ismatched, and steps S322 to S326 when all the matching is unmatched areperformed in the same manner as in the first example embodiment, thedescription thereof will be omitted.

When a warning is output in response to a failed identity verificationby face matching based on an image captured by the fixed camera 20 (stepS326), the staff of the facility uses the hand camera 110 of the gateapparatus 10 to capture the face of the visitor who is failed inidentity verification thereof (step S502). Note that, in the following,a face image of a face of a visitor captured by the hand camera 110 mayalso referred to as an in-hand face image.

Once an in-hand face image is captured by the hand camera 110, the gatecontrol unit 112 transmits image data of the in-hand face image to theface matching apparatus 30.

In the face matching apparatus 30, the face feature amount extractionunit 312 extracts a face feature amount of the transmitted in-hand faceimage.

Next, the face matching unit 314 matches the face feature amount of thein-hand face image extracted by the face feature amount extraction unit312 against a face feature amount of a registered face image registeredin association with ID information of the annual pass on ticketpresentation (step S504).

If the matching by the face matching unit 314 based on the in-hand faceimage is matched (step S506, YES), the process transfers to step S316 toopen the gate 106 (step S318). The visitor who made ticket presentationis able to walk through the path 114 of the gate apparatus 10 to enterthe inside of the facility as a person who is successful in identityverification by face matching based on the in-hand face image. Then, thegate 106 is closed (step S320).

In contrast, if the matching by the face matching unit 314 based on thein-hand face image is unmatched (step S506, NO), the visitor who madeticket presentation is attended by the staff of the facility as a personwho is failed in identity verification by face matching based on thein-hand face image (step S508). For example, as an action by the staffof the facility, identity verification is again done by the staff of thefacility.

As in the present example embodiment, when identity verification by facematching based on a detected face image by using the fixed camera 20 isfailed, identity verification by face matching based on an in-hand faceimage by using the hand camera 110 can be performed.

Note that, while the case where face matching based on an in-hand faceimage by using the hand camera 110 is performed in the face recognitionsystem according to the first example embodiment has been describedabove, the example embodiment is not limited thereto. The face matchingbased on an in-hand face image by using the hand camera 110 can beperformed in the face recognition system in the same manner as above inthe face recognition system according to the second to fourth exampleembodiment.

Sixth Embodiment

A face recognition system and a face recognition method according to asixth example embodiment of the present invention will be described byusing FIG. 19 and FIG. 20 . FIG. 19 is a flowchart illustrating the facerecognition method according to the present example embodiment. FIG. 20is a schematic diagram illustrating an example of a matching windowdisplayed on a display unit in the face recognition method according tothe present example embodiment. Note that components similar to those inthe face recognition system and the face recognition method according tothe first to fifth example embodiments described above will be labeledwith the same reference numeral, and the description thereof will beomitted or simplified.

In the present example embodiment, when identity verification by facematching based on an image captured by the fixed camera 20 is failed inthe face recognition system in the first example embodiment, aregistered face image and a detected face image are manually matchedthrough visual observation by the staff of the facility. The facerecognition method according to the present example embodiment will bedescribed below by using FIG. 19 and FIG. 20 .

Also in the present example embodiment, identity verification by facematching is performed based on an image captured by the fixed camera 20in the same manner as in the first example embodiment. As illustrated inFIG. 19 , since steps S302 to S314, steps S316 to S320 when matching ismatched, and steps S322 to S326 when all the matching is unmatched areperformed in the same manner as in the first example embodiment, thedescription thereof will be omitted.

When a warning is output in response to a failed identity verificationby face matching based on an image captured by the fixed camera 20 (stepS326), the face matching unit 314 displays a matching window on thedisplay unit 306 to the staff of the facility in the face matchingapparatus 30 (step S602). Specifically, the face matching unit 314displays a matching window on the display unit 306 in which a registeredface image and N detected face images on which the N:1 matching has beenperformed are displayed.

FIG. 20 illustrates an example of a matching window displayed on thedisplay unit 306. As illustrated in FIG. 20 , N detected face images inwhich matching by the face matching unit 314 is unmatched are displayedtogether with a registered face image in a matching window 322. Each ofthe N detected face images is determined that, as a result of matchingby the face matching unit 314, the matching is unmatched. As discussedabove, a registered face image and N detected face images in whichmatching by the face matching unit 314 is unmatched are displayed in thematching window 322 displayed on the display unit 306 so that the staffof the facility can perform visual matching.

The staff of the facility views the matching window displayed on thedisplay unit 306 to perform matching manually (step S604). Specifically,out of N detected face images displayed in the matching window, thestaff of the facility selects a detected face image which can bedetermined as the same person as a person of the registered face imagedisplayed together. The display unit 306 is formed of a touch panel andfunctions as an input unit, for example. In this case, the staff of thefacility may touch the detected face image displayed in the matchingwindow and input the selected result to the face matching apparatus 30.Note that a selected result may be input to the face matching apparatus30 by using other input units such as a mouse, a keyboard, or the like.

If the matching is matched as a result of manual matching (step S606,YES), that is, a detected face image which is determined as the sameperson as a person of the registered face image is selected, the processtransfers to step S316 to open the gate 106 (step S318). The visitor whothe made ticket presentation is able to walk through the path 114 of thegate apparatus 10 to enter the inside of the facility as a person who issuccessful in identity verification by face matching based on the manualmatching by the staff of the facility. Then, the gate 106 is closed(step S320).

On the other hand, if the matching is unmatched as a result of manualmatching (step S606, NO), that is, there is no detected face image whichis determined as the same person as a person of the registered faceimage, an action in which identity verification is again performed bythe staff of the facility or the like is taken (step S608).

The detected face image selected by the staff of the facility in themanual matching as described above can be registered as a new registeredface image. In this case, the face matching unit 314 of the facematching apparatus 30 transmits image data of the selected detected faceimage and the face feature amount thereof to the datacenter server 40.

In the datacenter server 40, the control unit 402 stores the transmitteddetected face image as a new registered face image in the storage unit404 in association with the ID information of the annual pass on theticket presentation. At this time, the control unit 402 also stores theface feature amount of the new registered face image in the storage unit404 in association with the ID information of the annual pass on theticket presentation.

As in the present example embodiment, when identity verification by facematching based on a detected face image by the fixed camera 20 isfailed, identity verification can be performed based on manual matchingthrough visual observation by the staff of the facility.

Note that, while the case where manual matching through visualobservation by the staff of the facility is performed in the facerecognition system according to the first example embodiment has beendescribed above, the example embodiment is not limited thereto. Themanual matching through visual observation by the staff of the facilitycan be performed in the face recognition system in the same manner asabove in the face recognition system according to the second to fifthexample embodiment.

Seventh Embodiment

A face recognition system and a face recognition method according to aseventh example embodiment of the present invention will be described byusing FIG. 21 . FIG. 21 is a block diagram illustrating a functionalconfiguration of the face recognition system of the present exampleembodiment. Note that components similar to those in the facerecognition system and the face recognition method according to thefirst to sixth example embodiments described above will be labeled withthe same reference numeral, and the description thereof will be omittedor simplified.

The basic configuration of the face recognition system according to thepresent example embodiment is substantially the same as theconfiguration of the face recognition system according to the firstexample embodiment. In the face recognition system according to thepresent example embodiment, a part of the information accumulated in thestorage unit 404 of the datacenter server 40 is stored in the facematching apparatus 30. In this regard, the face recognition systemaccording to the present example embodiment is different from the facerecognition system according to the first example embodiment.

As illustrated in FIG. 21 , a face recognition system 4 according to thepresent example embodiment further has a storage unit 324. Note that thestorage unit 324 may be formed of the same storage device as the storageunit 304.

The storage unit 324 stores registered face images registered inassociation with ID information of issued annual passes and the facefeature amounts thereof. The registered face images and the face featureamounts thereof stored in the storage unit 324 are a part of theregistered face images and the face feature amounts thereof accumulatedin the storage unit 404 of the datacenter server 40.

In the storage unit 324, a relational database is configured. In therelational database of the storage unit 324, face feature amounts ofregistered face images are stored in association with ID information ofannual passes and face image data of the registered face images, asdescribed above. Such mutually associated data is managed by an RDBMS.As an RDBMS, without being limited in particular, Microsoft (registeredtrademark) SQL Server is used, for example.

Also in the present example embodiment, in the same manner as in thefirst example embodiment, first, the face matching unit 314 attempts toacquire, online, a face feature amount of a registered face imageregistered in association with ID information of an annual pass onticket presentation from the datacenter server 40 via the network 60.

At this time, there may be a case where the face matching apparatus 30cannot be connected to the datacenter server 40 via the network 60 forsome reason such as the face matching apparatus 30 being offline. Whenthe face matching apparatus 30 cannot be connected to the datacenterserver 40, the face matching unit 314 cannot acquire, online, a facefeature amount of a registered face image from the datacenter server 40via the network 60.

In this case, the face matching unit 314 refers to the relationaldatabase in the storage unit 324 of the face matching apparatus 30 andacquires, offline, the face feature amount of the registered face imageregistered in association with the ID information of the annual pass onthe ticket presentation.

The face matching unit 314 can use a registered face image acquired fromthe storage unit 324 of the face matching apparatus 30 to perform N:1matching in the same manner as the first example embodiment

As discussed above, according to the present example embodiment, theface matching apparatus 30 has the storage unit 324 offline in whichregistered face images registered in association with ID information ofthe annual passes and the face feature amounts thereof are stored.Therefore, according to the present example embodiment, even when facefeature amounts of registered face images cannot be acquired from thedatacenter server 40 online, face matching can be performed.

Eighth Example Embodiment

A face recognition system and a face recognition method according to aneighth example embodiment of the present invention will be described byusing FIG. 22 to FIG. 24 . Note that components similar to those in theface recognition system and the face recognition method according to thefirst to seventh example embodiments described above will be labeledwith the same reference numeral, and the description thereof will beomitted or simplified.

First, the face recognition system according to the present exampleembodiment will be described by using FIG. 22 and FIG. 23 . FIG. 22 is aschematic diagram illustrating the face recognition system according tothe present example embodiment. FIG. 23 is a block diagram illustratinga functional configuration of the face recognition system according tothe present example embodiment.

In the above first to eighth embodiments, the case of performingidentity verification by face matching on a visitor as an authenticationsubject who is going to enter the inside of the facility by using anadmission ticket at an entrance gate of a facility has been described.However, a situation of performing identity verification by facematching is not limited thereto. In the present example embodiment, acase of performing identity verification by face matching on a shoppingcustomer as an authentication subject who performs electronic payment byusing a credit card, a debit card, electronic money, or the like at acashier area such as a register counter where a register terminal isinstalled in a shop will be described.

As illustrated in FIG. 22 and FIG. 23 , a face recognition system 5according to the present example embodiment includes a register terminal810, the fixed camera 20, the face matching apparatus 30, the datacenterserver 40, and an electronic payment server 820. The register terminal810, the fixed camera 20, and the face matching apparatus 30 areinstalled at a casher area 850 of a shop. On the other hand, thedatacenter server 40 is installed inside a datacenter remote from thecasher area 850, for example. Further, the electronic payment server 820is a server of a company that provides an electronic payment system, forexample.

The face matching apparatus 30 and the datacenter server 40 areconnected to a network via the network 60, respectively, and cancommunicate with each other via the network 60 in the same manner as inthe first example embodiment.

Further, the register terminal 810 and the electronic payment server 820are connected to a network via the network 60, respectively, and cancommunicate with each other via the network 60.

Further, the register terminal 810 and the fixed camera 20 are directly,locally connected to the face matching apparatus 30 in a communicablemanner through cable connection or the like, respectively. Theconnection among the register terminal 810, the fixed camera 20, and theface matching apparatus 30 may be of a wired scheme or a wirelessscheme.

Next, each unit of the face recognition system 5 in the present exampleembodiment will be described in detail.

The register terminal 810 reads price information of a purchase itempurchased by a shopping customer as well as other information andperforms an accounting process for the purchase item. As describedlater, the register terminal 810 performs an accounting process based onelectronic payment information notified when identity verification issuccessful by face matching.

A reading unit 830 is connected to the register terminal 810. Thereading unit 830 is installed near the register terminal 810 adjacent tothe register terminal 810. The reading unit 830 reads informationrecorded in a membership card of a shop carried by a shopping customer.Specifically, in a membership card, ID information that uniquelyidentifies the membership card is recorded. The reading unit 830 readsID information from a membership card. The ID information read by thereading unit 830 is a membership number of the membership card, forexample. A shopping customer using a shop carries a membership card whenusing the shop and causes the reading unit 830 to read the membershipcard in accounting. The membership card is a medium that is carried by ashopping customer, which is an authentication subject, and required whenthe shopping customer makes a payment and in which ID informationuniquely identifying this is recorded. As described later, in thedatacenter server 40, information on registered members to whichmembership cards are issued in association with ID information on themembership cards is accumulated.

The reading unit 830 has a reading scheme in accordance with a recordingscheme of ID information of a membership card. For example, when amembership card has ID information recorded in a one-dimensional codesuch as a barcode or a two-dimensional code such as a QR code(registered trademark), the reading unit 108 is a code reader such as abarcode reader, a QR code reader, or the like. Further, for example,when a membership card has ID information recorded in a non-contact ICcard or a non-contact IC tug with Radio Frequency Identification (RFID),the reading unit 830 is an RFID reader.

When there is ticket presentation of a membership card on the readingunit 830, the reading unit 830 reads ID information recorded in themembership card from the membership card. Note that ticket presentationhere means that a shopping customer who is an authentication subjectcauses the reading unit 830 to read information including the IDinformation recorded in a membership card.

The reading unit 830 transmits ID information read from a membershipcard to the register terminal 810. The register terminal 810 transmits,to the face matching apparatus 30, the ID information of the membershipcard transmitted from the reading unit 830.

Further, the register terminal 810 performs an accounting process basedon the electronic payment information transmitted from the face matchingapparatus 30 described later.

The fixed camera 20 is fixed to the upper end of a support pillar 202installed on the exit side of the cashier area 850 with respect to theregister terminal 810. The fixed camera 20 captures an image of an areain front of the register terminal 810, and the orientation facing theentrance side of the cashier area 850 is fixed. The fixed camera 20 isfixed at a height located above a head of a human of a height of around200 cm, for example, from the ground face at the cashier area 850 and isdirected obliquely downward to face an area in front of the registerterminal 810. Note that a fixing scheme of the fixed camera 20 is notlimited to a scheme using the support pillar 202. For example, the fixedcamera 20 may be hanged from and fixed to the ceiling.

The fixed camera 20 fixed as described above captures an image of anarea in front of the register terminal 810 that is the entrance side tothe installation area of the register terminal 810 and the reading unit830. Thereby, the fixed camera 20 can capture a shopping customer C inthe area in front of the register terminal 10 that is the entrance sideto the installation area of the reading unit 830.

Since other features of the fixed camera 20 is the same as those in theabove first example embodiment, the description thereof will be omitted.

The face matching apparatus 30 has the face matching control unit 302,the storage unit 304, and the display unit 306 in the same manner as thefirst example embodiment. The face matching control unit 302 includesthe image data acquisition unit 308, the face detection unit 310, theface feature amount extraction unit 312, and the face matching unit 314.

The image data acquisition unit 308 sequentially acquires image data ofimages transmitted from the fixed camera 20 at a predetermined cycle inthe same manner as the first example embodiment.

The face detection unit 310 performs face detection on respective imagesof image data sequentially acquired from the image data acquisition unit308 in the same manner as the first example embodiment. Note that, inthe present example embodiment, the face detection unit 310 detects aface image of a shopping customer in an area in front of the registerterminal 810.

The face feature amount extraction unit 312 extracts a face featureamount for respective detected face images detected by the facedetection unit 310 in the same manner as the first example embodiment.Further, the face feature amount extraction unit 312 temporarily storesface image data, a face feature amount, a detection number, and acapturing time in the storage unit 304 in association with each otherfor respective detected face images in the same manner as the firstexample embodiment.

A relational database is configured in the storage unit 304, and facefeature amounts and data related thereto are temporarily stored for acertain time period from the capturing time for respective detected faceimages in the same manner as the first example embodiment.

In response to ticket presentation of a membership card at the readingunit 830 connected to the register terminal 810, the face matching unit314 performs identity verification by face matching for a shoppingcustomer who made ticket presentation of the membership card at thereading unit 830.

ID information read by the reading unit 830 from the membership card onticket presentation is transmitted to the face matching unit 314. Theface matching unit 314 acquires the transmitted ID information and, viathe network 60 from the datacenter server 40 described later, acquires aface feature amount of a registered face image registered in associationwith the ID information. A person of a registered face image acquired bythe face matching unit 314 in such a way is a valid user who is allowedto validly use the membership card on ticket presentation. A valid userof a membership card is a registered member to which the membership cardwas issued, for example.

Further, in the same manner as the first example embodiment, the facematching unit 314 refers to the relational database of the storage unit304 and acquires face feature amounts of N detected face imagesassociated with the capturing time included in a predetermined periodbefore ticket presentation. That is, the face matching unit 314 acquiresface feature amounts of N detected face images captured by the fixedcamera 20 before the reading unit 830 reads ID information from amembership card.

The face matching unit 314 performs N:1 matching that sequentiallymatches respective face feature amounts of N detected face images, whichhave been captured before ticket presentation of a membership card,against a face feature amount of a registered face image in the samemanner as the first example embodiment.

If matching by the face matching unit 314 is matched, this means that avalid user of a membership card on ticket presentation is in shoppingcustomers in front of the register terminal 810 before the ticketpresentation. Thus, it can be estimated that a valid user of amembership card made the ticket presentation of the membership card.Therefore, in this case, identity verification by face matching issuccessful.

On the other hand, if all the matching by the face matching unit 314 isunmatched, this means that there is no valid user of a membership cardon ticket presentation in shopping customers in front of the registerterminal 810 before the ticket presentation. Therefore, in this case,identity verification by face matching is failed.

A matching result or the like by the face matching unit 314 can bedisplayed on the display unit 306. The staff of the shop can confirm amatching result or the like by viewing the display on the display unit306.

The face matching unit 314 transmits, to the register terminal 810, amatching result signal that is a signal indicating the matching resultdescribed above. Specifically, the face matching unit 314 transmits, tothe register terminal 810, a matching-matched signal that is a signalindicating that the matching performed by the face matching unit 314 ismatched or a matching-unmatched signal that is a signal indicating thatall the matching performed by the face matching unit 314 is unmatched.

Furthermore, if the matching is matched, the face matching unit 314acquires payment option information via the network 60 from thedatacenter server 40 described later. The payment information isinformation used for performing an electronic payment stored inassociation with ID information of a membership card in which matchingis matched and may be a credit card number or electronic money IDinformation, for example. The face matching unit 314 transmits theacquired payment option information to the register terminal 810together with the matching-matched signal.

When a matching-matched signal is transmitted from the face matchingunit 314, the register terminal 810 requests an electronic payment tothe electronic payment server 820 described later. In this case, theregister terminal 810 transmits, to the electronic payment server 820,electronic payment information that is information including the paymentoption information transmitted together with the matching-matched signaland an accounting price of a purchased item.

On the other hand, when a matching-unmatched signal is transmitted fromthe face matching unit 314, the register terminal 810 uses a warningdisplay, a warning sound, or the like to notify the shop staff operatingthe register terminal 810 that the electronic payment cannot be made.

The datacenter server 40 has the control unit 402 and the storage unit404 in the same manner as the first example embodiment.

The storage unit 404 accumulates registered face images registered inassociation with ID information of issued membership cards and facefeature amounts thereof. Furthermore, in the storage unit 404, paymentoption information associated with ID information of membership cards isstored.

In response to a request from the face matching unit 314, the controlunit 402 provides, to the face matching unit 314, a face feature amountof a registered face image registered in association with the IDinformation of the membership card on ticket presentation. Furthermore,in response to a request from the face matching unit 314, the controlunit 402 provides, to the face matching unit 314, the payment optioninformation associated with the ID information of the membership card onthicket presentation.

A registered face image can be uploaded to the datacenter server 40 froma terminal in a shop, a terminal of a shopping customer, or the likewhen the shopping customer acquires a membership card of the shop, forexample. In the datacenter server 40, the control unit 402 accumulatesthe uploaded registered face image in the storage unit 404.

For a registered face image accumulated in the storage unit 404 of thedatacenter server 40 as described above, a face feature amount isextracted and accumulated in the storage unit 404 in the same manner asthe first example embodiment.

A relational database is configured in the storage unit 404, and facefeature amounts of registered face images are stored in association withID information of membership cards and face image data of the registeredface images in the same manner as the first example embodiment.Furthermore, in the storage unit 404, payment option information isstored in association with ID information of membership cards.

Note that, in the relational database of the storage unit 404, forexample, information on names, contact addresses, or the like ofregistered members who are valid users of membership cards is stored inaddition to the above in association with ID information of membershipcards.

The electronic payment server 820 performs an electronic payment by acredit card, an electronic money, or the like. The electronic paymentserver 820 performs an electronic payment for an item purchased by ashopping customer based on the electronic payment informationtransmitted from the register terminal 810.

As described above, the face recognition system 5 according to thepresent example embodiment matches, against a registered face imageregistered in association with the ID information of the membership cardon the ticket presentation, detected face images captured by the fixedcamera 20 before ticket presentation in which the reading unit 830 readsID information from a membership card is performed. That is, in the facerecognition system 5 according to the present example embodiment, thedetected face image that is an image of a matching subject to be matchedagainst a registered face image is acquired in advance before ticketpresentation of a membership card.

Thus, according to the present example embodiment, after a shoppingcustomer makes ticket presentation of a membership card, it is notnecessary to capture a face image of the shopping customer as an imageof a matching subject to be matched against a registered face image bythe staff of a shop. Further, the shopping customer neither needs toconcern about capturing of the face image thereof nor needs to perform aspecial move such as positioning of the face thereof for the capturing.Therefore, according to the present example embodiment, face matchingcan be made smoothly in a short time.

Next, a face recognition method according to the present exampleembodiment using the face recognition system 5 according to the presentexample embodiment will be further described by using FIG. 24 . FIG. 24is a flowchart illustrating the face recognition method according to thepresent example embodiment.

The fixed camera 20 captures an area in front of the register terminal810 that is the entrance side to the installation area of the readingunit 830 and acquires a plurality of images continuously at apredetermined cycle. Since the process before temporality storing a facefeature amount of a detected face image from the capturing by the fixedcamera 20 is the same as the process illustrated in FIG. 9 of the firstexample embodiment except that the capturing area by the fixed camera 20is different, the description thereof will be omitted.

A process from the ticket presentation of a membership card to executionof an electronic payment will be described below by using FIG. 24 .

At the register terminal 810, the reading unit 830 stands by untilticket presentation of a membership card is performed (step S702, NO).

In response to ticket presentation of the membership card at the readingunit 830 by a shopping customer (step S702, YES), the reading unit 830reads ID information recorded in the membership card from the membershipcard (step S704).

Subsequently, the register terminal 810 transmits the ID informationread by the reading unit 830 to the face matching apparatus 30 (stepS706).

In the face matching apparatus 30, the face matching unit 314 transmits,to the datacenter server 40 via the network 60, the ID informationtransmitted from the register terminal 810 and requests a face featureamount of a registered face image. Thereby, from the datacenter server40 via the network 60, the face matching unit 314 acquires, online, aface feature amount of a registered face image registered in associationwith the transmitted ID information (step S708).

Further, the face matching unit 314 refers to the relational database ofthe storage unit 304 to acquire, offline, a face feature amount of Ndetected face images associated with the capturing time included in apredetermined period before the ticket presentation of the membershipcard (step S710). That is, the face matching unit 314 acquires facefeature amounts of N detected face images captured by the fixed camera20 before the reading unit 830 reads ID information from the membershipcard.

The face matching unit 314 performs N:1 matching that sequentiallymatches respective face feature amounts of the acquired N detected faceimages against the face feature amount of the acquired registered faceimage in the same manner as the first example embodiment (step S712).

If the matching performed by the face matching unit 314 is matched (stepS714, YES), the face matching unit 314 acquires payment optioninformation from the datacenter server 40 via the network 60 (stepS716). Subsequently, the face matching unit 314 transmits, to theregister terminal 810 together with the acquired payment optioninformation, a matching-matched signal indicating that the matchingperformed by the face matching unit 314 is matched (step S718).

In response to the transmission of the matching-matched signal from theface matching unit 314, the register terminal 810 transmits, to theelectronic payment server 820, electronic payment information includingthe transmitted payment option information and an accounting price of apurchased item (step S720).

The electronic payment server 820 executes the electronic payment forthe item purchased by the shopping customer based on the electronicpayment information transmitted from the register terminal 810 (stepS722).

On the other hand, if all the matching performed by the face matchingunit 314 is unmatched (step S714, NO), the face matching unit 314transmits, to the register terminal 810, a matching-unmatched signalindicating that all the matching performed by the face matching unit 314is unmatched (step S724).

In response to the transmission of the matching-unmatched signal fromthe face matching unit 314, the register terminal 810 uses a warningdisplay, a warning sound, or the like to notify the shop staff operatingthe register terminal 810 that the electronic payment cannot be made(step S726).

The process illustrated in FIG. 24 described above is repeatedlyperformed every time ticket presentation is made at the reading unit 830connected to the register terminal 810 during the face recognitionmethod according to the present example embodiment being performed.

As discussed above, according to the present example embodiment, a facefeature amount of a detected face image detected from an image capturedby the fixed camera 20 before ticket presentation of a membership cardis matched against a face feature amount of a registered face imageregistered in association with the ID information of the membership cardon the ticket presentation. Therefore, according to the present exampleembodiment, face matching can be performed smoothly in a short time.

Note that, also when identity verification is performed by face matchingat a cashier area where a register terminal is installed in the shopdescribed above, the face recognition system may be configured in thesame manner as the second to seventh example embodiments described aboveto perform the same process as that in the second to seventh exampleembodiments.

Ninth Example Embodiment

A computer apparatus according to the ninth example embodiment of thepresent invention will be described by using FIG. 25 . In the presentexample embodiment, a computer apparatus used for implementing processesof respective units of the face recognition system according to thefirst to eighth example embodiments described above will be described.

FIG. 25 illustrates an example of a computer apparatus used forimplementing processes of respective units of the face recognitionsystem according to the first to eighth example embodiments describedabove will be described. A computer apparatus 1000 illustrated in FIG.25 is not limited in particular but may be of various types or forms.For example, the computer apparatus 1000 may be a laptop computer, adesktop computer, a workstation, a personal digital assistant, a server,a blade server, a main frame, an embedded system, or the like.

The computer apparatus 1000 has a processor 1002, memory 1004, and astorage device 1006. Further, the computer apparatus 1000 has ahigh-speed controller 1008 including a high-speed interface and alow-speed controller 1010 including a low-speed interface. The memory1004 and a high-speed expansion port 1012 are connected to thehigh-speed controller 1008. Further, an external input/output devicesuch as a display 1016 or the like is connected to the high-speedcontroller 1008. On the other hand, a low-speed expansion port 1014 andthe storage device 1006 are connected to the low-speed controller 1010.

The processor 1002, the memory 1004, the storage device 1006, thehigh-speed controller 1008, the low-speed controller 1010, and thehigh-speed expansion port 1012 are connected to each other throughvarious buses. Further, the processor 1002, the memory 1004, the storagedevice 1006, the high-speed controller 1008, the low-speed controller1010, and the high-speed expansion port 1012 may be implemented on acommon motherboard or may be implemented in other forms as appropriate.

The processor 1002 is a central processing unit (CPU), for example, andis able to process instructions executed within the computer apparatus1000. Such instructions include an instruction that is used fordisplaying graphics information of a graphical user interface (GUI) onan external input/output device such as the display 1016 and stored inthe memory 1004 or the storage device 1006.

Further, a plurality of processors, a plurality of busses, or aplurality of processors and a plurality of busses can be used asappropriate together with a plurality of memory devices and multipletypes of memory devices. Further, a plurality of computer apparatus 1000can be connected to each device that performs a part of the necessaryprocess. For example, a plurality of computer apparatus 1000 can beconnected to each other as a server bank, a group of blade servers, or amultiprocessor system.

The memory 1004 stores therein information within the computer apparatus1000. For example, the memory 1004 may be a volatile memory unit or anon-volatile memory unit. The memory 1004 may be another computerreadable medium, such as a magnetic disk, an optical disk, or the like,for example.

The storage device 1006 can configure mass storage used for the computerapparatus 1000. The storage device 1006 may be, for example, a computerreadable medium such as a floppy (registered trademark) disk device, ahard disk device, an optical disk device, a tape device, a solid statememory device such as a flash memory, a disk array, or the like.Further, the storage device 1006 may include such a computer readablestorage medium. The storage device 1006 may include a storage areanetwork or a device with another configuration. A computer programproduct may be tangibly embodied in an information carrier. The computerprogram product can also store an instruction that executes one or aplurality of processes as described above when executed. The informationcarrier may be a memory device such as the memory 1004, the storage unit1006, or the memory on the processor 1002 or may be a computer readablemedium or a machine readable medium such as a carrier signal.

The high-speed controller 1008 manages processes in which the bandwidthfor the computer apparatus 1000 is intensively used. On the other hand,the low-speed controller 1010 manages processes in which the bandwidthis less intensively used. However, such allocation of the functions is amere example, and allocation is not limited thereto. Further, a part ora whole of the high-speed controller 1008 may be incorporated in theprocessor 1002.

The high-speed controller 1008 is connected to the high-speed expansionport 1012 that can accept the memory 1004 and various expansion cards.Further, the high-speed controller 1008 is connected to the display 1016via a graphics processor or an accelerator, for example.

Further, the low-speed controller 1010 is connected to the storagedevice 1006 and the low-speed expansion port 1014. The low-speedexpansion port 1014 can include, for example, a communication port ofvarious standards such as Universal Serial Bus (USB), Bluetooth(registered trademark), wired or wireless Ethernet (registeredtrademark), or the like. One or plurality of input/output devices suchas a keyboard, a pointing device, a scanner, or the like can beconnected to the low-speed expansion port 1014. Further, one orplurality of network devices such as a switch, a router, or the like canbe connected to the low-speed expansion port 1014 via a network adapter,for example.

The computer apparatus 1000 can be implemented in many different forms.For example, the computer apparatus 1000 can be implemented in a form ofa typical server or a plurality of servers in a form of a group of suchservers. Further, the computer apparatus 1000 can be implemented as apart of the rack server system. Furthermore, the computer apparatus 1000can be implemented in a form of a personal computer such as a laptopcomputer, a desktop computer, or the like.

The computer apparatus 1000 described above can function as a part ofthe gate apparatus 10 in the example embodiments described above. Inthis case, the processor 1002 of the computer apparatus 1000 canfunction as the gate control unit 112 by executing a program thatimplements the function of the gate control unit 112 of the gateapparatus 10.

Further, the computer apparatus 1000 can function as the face matchingapparatus 30 in the example embodiments described above. In this case,the processor 1002 of the computer apparatus 1000 can function as theface matching control unit 302 by executing a program that implementsthe function of the face matching control unit 302 of the face matchingapparatus 30. That is, the processor 1002 executes programs thatimplement the functions of respective units of the image dataacquisition unit 308, the face detection unit 310, the face featureamount extraction unit 312, the face matching unit 314, the prioritycalculation unit 318, and the identical-person processing unit 320.Thereby, the processor 1002 can function as each unit of the image dataacquisition unit 308, the face detection unit 310, the face featureamount extraction unit 312, the face matching unit 314, the prioritycalculation unit 318, and the identical-person processing unit 320.Further, the storage device 1006 of the computer apparatus 1000 canfunction as the storage units 304 and 324 of the face matching apparatus30.

The computer apparatus 1000 can function as the datacenter server 40 inthe example embodiments described above. In this case, the processor1002 of the computer apparatus 1000 can function as the control unit 402by executing a program that implements the function of the control unit402 of the datacenter server 40. Further, the storage device 1006 of thecomputer apparatus 1000 can function as the storage unit 404 of thedatacenter server 40.

Note that a part or a whole of the program executed by the processor1002 of the computer apparatus 1000 can be provided by a computerreadable storage medium storing the above, such as a digital versatiledisc-read only memory (DVD-ROM), a compact disc-read only memory(CD-ROM), a flash memory such as a USB memory or the like.

Other Example Embodiments

The face recognition system illustrated in each of the exampleembodiments described above can be configured as illustrated in FIG. 26according to another example embodiment. FIG. 26 is a block diagramillustrating a functional configuration of a face recognition systemaccording to another example embodiment.

As illustrated in FIG. 26 , a face recognition system 5000 according toanother example embodiment has a reading unit 5002 that readsidentification information from a medium carried by an authenticationsubject and an image capturing unit 5004 that captures an entrance sideto an installation area of the reading unit 5002 to acquire an image.Further, the face recognition system 5000 has a face detection unit 5006that detects a face image as a detected face image from the imageacquired from the image capturing unit 5004. Furthermore, the facerecognition system 5000 has a face matching unit 5008 that matches thedetected face image detected by the face detection unit 5006 against aregistered face image registered in association with the identificationinformation read by the reading unit 5002. The face matching unit 5008matches, against the registered face image, the detected face imagecaptured by the image capturing unit 5004 before the reading unit 5002reads the identification information.

Further, the face matching apparatus described in each of the aboveexample embodiments can be configured as illustrated in FIG. 27according to another example embodiment. FIG. 27 is a block diagramillustrating a functional configuration of a face matching apparatusaccording to another example embodiment.

As illustrated in FIG. 27 , a face matching apparatus 6000 according toanother example embodiment has a face detection unit 6002 that detects aface image from an image as a detected face image. The image by whichthe face detection unit 6002 detects the detected face image is acquiredby an image capturing unit that captures and acquires an image of theentrance side to an installation area of a reading unit that readsidentification information from an authentication subject. Furthermore,the face matching apparatus 6000 has a face matching unit 6004 thatmatches the detected face image detected by the face detection unit 6002against a registered face image registered in association with theidentification information read by the reading unit. The face matchingunit 6004 matches the detected face image captured by the imagecapturing unit before the reading unit reads the identificationinformation against the registered face image.

Further, the face recognition system described in each of the aboveexample embodiments can be configured as illustrated in FIG. 28according to yet another example embodiment. FIG. 28 is a block diagramillustrating a functional configuration of a face recognition systemaccording to yet another example embodiment.

As illustrated in FIG. 28 , a face recognition system 7000 according toyet another example embodiment has a face detection unit 7002 thatdetects a face image from an image including an authentication subjectas a detected face image. Further, the face recognition system 7000 hasa storage unit 7004 that stores identification information identifyingthe authentication subject in association with a registered face imageof the authentication subject. Furthermore, the face recognition system7000 has a face matching unit 7006 that, in response to acquiring theidentification information identifying the authentication subject,matches, against the registered face image corresponding to the acquiredidentification information, the detected face image detected by the facedetection unit 7002 from the image captured before the acquisition.

Modified Example Embodiments

The present invention is not limited to the example embodimentsdescribed above, and various modifications are possible.

For example, while the situations where identity verification by facematching is performed for visitors who intend to enter the inside of thefacility as authentication subjects and for shopping customers who makeelectronic payment as an authentication subject, respectively, have beendescribed as examples in the above example embodiments, the invention isnot limited thereto. An authentication subject refers to a person to beverified as to whether or not to have some authority, such as a visitorentering a facility, an entrant at immigration examination, or the like.A situation where identity verification by face matching may be asituation where identity verification by face matching is performed inimmigration control, entrance and exit control for a room, or the like,for example.

Further, while the cases where the fixed camera 20 is used as an imagecapturing unit have been described as examples in the above exampleembodiments, a camera that functions as the image capturing unit is notlimited thereto. For example, instead of the fixed camera 20, a movablecamera in which the orientation thereof can be changed by a panfunction, a tilt function, or the like may be used as the imagecapturing unit. In the case of a movable camera, the orientation thereofcan be changed by automatic control or remote control.

Further, while the cases where identity verification by face matching isperformed after ticket presentation is made by an authentication subjectwhose annual pass or membership card information is read by the readingunit 108 or 830 have been described as examples in the above exampleembodiments, the invention is not limited thereto. For example, thereading units 108 and 830 can be configured to read, from a mediumcarried by an authentication subject, identification information such asID information uniquely identifying the medium by using wirelesscommunication or the like without requiring action, namely, ticketpresentation by an authentication subject.

The whole or part of the example embodiments disclosed above can bedescribed as, but not limited to, the following supplementary notes.

(Supplementary Note 1)

A face recognition system comprising:

a face detection unit that detects a face image from an image includingan authentication subject as a detected face image;

a storage unit that stores identification information identifying theauthentication subject and a registered face image of the authenticationsubject in association with each other; and

a face matching unit that, in response to acquisition of theidentification information identifying the authentication subject,matches, against the registered face image corresponding to the acquiredidentification information, the detected face image detected by the facedetection unit from an image captured before the acquisition.

(Supplementary Note 2)

The face recognition system according to supplementary note 1,

wherein the face detection unit detects a plurality of detected faceimages from a plurality of frames of images, and

wherein the face matching unit matches the plurality of detected faceimages against the registered face image.

(Supplementary Note 3)

The face recognition system according to supplementary note 2 furthercomprising a priority calculation unit that calculates a priority usedfor determining order of performing matching against the registered faceimage for the plurality of detected face images,

wherein the face matching unit matches the plurality of detected faceimages against the registered face image in descending order of thepriority calculated by the priority calculation unit.

(Supplementary Note 4)

The face recognition system according to supplementary note 2 or 3further comprising an identical-person processing unit that classifiesthe plurality of detected face images on an identical person basis andcalculates a priority for determining order of performing matchingagainst the registered face image for the detected face image classifiedinto the identical person,

wherein the face matching unit matches the detected face imageclassified into the identical person against the registered face imagein descending order of the priority calculated by the identical-personprocessing unit.

(Supplementary Note 5)

The face recognition system according to any one of supplementary notes1 to 4 further comprising an update unit that registers, as a newregistered face image, the detected face image in which matching againstthe registered face image is matched.

(Supplementary Note 6)

The face recognition system according to any one of supplementary notes1 to 5 further comprising:

an image capturing unit that captures an image including theauthentication subject; and

another image capturing unit that captures a face image of theauthentication subject and acquires the face image of the authenticationsubject when matching of the detected face image against the registeredface image is not matched,

wherein the face matching unit matches, against the registered faceimage, the face image of the authentication subject acquired by theanother image capturing unit.

(Supplementary Note 7)

The face recognition system according to any one of supplementary notes1 to 5 further comprising a display unit that displays the detected faceimage and the registered face image when matching of the detected faceimage and the registered face image is not matched.

(Supplementary Note 8)

The face recognition system according to any one of supplementary notes1 to 7, wherein the face matching unit acquires, offline, the registeredface image to be matched against the detected face image.

(Supplementary Note 9)

The face recognition system according to any one of supplementary notes1 to 8 further comprising an image capturing unit that captures an imageincluding the authentication subject,

wherein the image capturing unit is installed in a vertical orientationso as to capture the image that is vertically long.

(Supplementary Note 10)

A face recognition method comprising:

detecting a face image from an image including an authentication subjectas a detected face image; and

in response to acquisition of the identification information identifyingthe authentication subject, matching, against a registered face imageregistered in association with the acquired identification information,the detected face image detected from an image captured before theacquisition.

(Supplementary Note 11)

The face recognition method according to supplementary note 10 furthercomprising:

detecting a plurality of detected face images from a plurality of framesof images; and

matching the plurality of detected face images against the registeredface image.

(Supplementary Note 12)

A storage medium in which a program is stored, wherein the programcauses a computer to execute:

detecting a face image from an image including an authentication subjectas a detected face image; and

in response to acquisition of the identification information identifyingthe authentication subject, matching, against a registered face imageregistered in association with the acquired identification information,the detected face image detected from an image captured before theacquisition.

While the present invention has been described with reference to theexample embodiments, the present invention is not limited to the exampleembodiment described above. Various modification that can be understoodby those skilled in the art can be made to the configuration or thedetails of the present invention within the scope of the presentinvention.

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2016-036406, filed on Feb. 26, 2016, thedisclosure of which is incorporated herein in its entirety by reference.

REFERENCE SIGNS LIST

-   1, 2, 3, 4, 5 face recognition system-   10 gate apparatus-   20 fixed camera-   30 face matching apparatus-   40 datacenter server-   108 reading unit-   110 hand camera-   112 gate control unit-   302 face matching control unit-   304 storage unit-   308 image data acquisition unit-   310 face detection unit-   312 face feature amount extraction unit-   314 face matching unit-   318 priority calculation unit-   320 identical-person processing unit-   324 storage unit-   402 control unit-   404 storage unit-   810 register terminal-   830 reading unit

1. A face recognition system comprising: a memory configured to storeinstructions; and a processor configured to execute the instructions to:detect a plurality of faces from a plurality of images captured by acamera, each of the plurality of faces corresponding to a differentperson including an authentication subject; acquire, from a database inwhich a plurality of registered images are stored, a registered faceimage; identify a face image for matching from the image including thedetected face based on quality; and compare a plurality of theregistered face images with the face image for matching, wherein theimage is captured before the authentication subject arrives at a gate.2. The face recognition system according to claim 1, wherein theprocessor is further configured to execute the instructions to:calculate the quality of the image including the detected face.
 3. Theface recognition system according to claim 1, wherein the processor isfurther configured to execute the instructions to: acquire, from thedatabase in which the plurality of registered images are stored, animage to which identification information is related as the registeredface image.
 4. The face recognition system according to claim 1, whereinthe image is captured before identification information is presented. 5.The face recognition system according to claim 1, wherein the face imagefor matching is captured in a predetermined period among the detectedfaces.
 6. The face recognition system according to claim 1, wherein theimage is captured before the authentication subject makes a payment, andwherein the processor is further configured to execute the instructionsto: identify the face image for matching from the image including thedetected face based on the quality; and perform the payment based onpayment information corresponding a result of face matching and theregistered face image.
 7. A face recognition system comprising: a memoryconfigured to store instructions; a processor configured to execute theinstructions to: detect a plurality of faces from a plurality of imagescaptured by a camera, each of the plurality of faces corresponding to adifferent person including an authentication subject; acquire, from adatabase in which a plurality of registered images are stored, aregistered face image; identify a face closest to a gate from thedetected faces as a face image for matching; and compare a plurality ofthe registered face images with the face image for matching, wherein theimage is captured before the authentication subject arrives at the gate.8. The face recognition system according to claim 7, wherein theprocessor is further configured to execute the instructions to: identifya largest face in the plurality of images as the face image formatching.
 9. The face recognition system according to claim 7, whereinthe processor is further configured to execute the instructions to:acquire, from the database in which the plurality of registered imagesare stored, an image to which identification information is related asthe registered face image.
 10. The face recognition system according toclaim 7,wherein the image is captured before identification informationis presented.
 11. The face recognition system according to claim 7,wherein the face image for matching is captured in a predeterminedperiod among the detected faces.
 12. The face recognition systemaccording to claim 7, wherein the image is captured before theauthentication subject makes a payment, and wherein the processor isfurther configured to execute the instructions to: identify the faceclosest to the gate from the detected faces as the face image formatching; and perform the payment based on the payment informationcorresponding the result of the face matching and the registered faceimage.
 13. A face recognition method comprising: detect a plurality offaces from a plurality of images captured by a camera, each of theplurality of faces corresponding to a different person including anauthentication subject; acquire, from a database in which a plurality ofregistered images are stored, a registered face image; identify a faceimage for matching from the image including the detected face based onquality; and compare a plurality of the registered face images with theface image for matching, wherein the image is captured before theauthentication subject arrives at a gate.
 14. The face recognitionmethod according to claim 13, wherein the processor is furtherconfigured to execute the instructions to: calculate the quality of theimage including the detected face.
 15. The face recognition systemaccording to claim 13, wherein the processor is further configured toexecute the instructions to: acquire, from the database in which theplurality of registered images are stored, an image to whichidentification information is related as the registered face image. 16.The face recognition system according to claim 13, wherein the image iscaptured before identification information is presented.
 17. The facerecognition system according to claim 13, wherein the face image formatching is captured in a predetermined period among the detected faces.18. The face recognition system according to claim 13, wherein the imageis captured before the authentication subject makes a payment, andwherein the processor is further configured to execute the instructionsto: identify the face image for matching from the image including thedetected face based on the quality; and perform the payment based onpayment information corresponding a result of face matching and theregistered face image.